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ABSTRACT 


The  objective  of  this  thesis  is  to  estimate  Inter- 
Deployment  Training  Cycle  (IDTC)  performances  of  US  Pacific 
Fleet  surface  ships,  which  are  evaluated  at  the  end  of  the 
Basic  Training  Phase,  by  using  Command  Metrics  Tool  (COMET) 
metrics.  The  objective  was  primarily  to  decide  whether  the 
COMET  database  can  be  used  to  estimate  the  performances  of 
ships,  and  to  build  regression  models  to  estimate  Einal 
Evaluation  Problem  (EEP)  performances  of  ships. 

This  study  develops  multivariate  logit  regression 
models  to  examine  and  explore  the  structure  of  the  data 
sets.  Most  of  the  models  developed  according  to  statistical 
criteria  include  only  the  intercept,  indicating  that  there 
is  no  real  relationship  between  the  COMET  metrics  and  IDTC 
performances.  The  assessments  made  at  the  end  of  EEP  are 
not  good  Measure  of  Performances  (MOPs)  by  which  to  assess 
ships'  IDTC  performances. 
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EXECUTIVE  SUMMARY 


It  is  important  for  the  US  Navy  to  make  its  ships 
ready  for  battle  and  deployment  in  a  short  period  of  time. 
After  finishing  their  deployment,  ships  undergo  an  Inter- 
Deployment  Training  Cycle  (IDTC),  the  aim  of  which  is  to 
make  ships  ready  for  their  next  deployment. 

If  areas  that  might  make  the  ship  fail  the  Final 
Evaluation  Problem  (FEP)  in  the  Basic  Training  Phase  could 
be  determined  before  the  FEP  takes  place,  the  ship  would 
concentrate  on  these  areas  before  encountering  problems.  As 
a  result,  this  would  increase  a  ship's  probability  of 
passing  the  FEP. 

The  Command  Metrics  Tools  (COMET)  is  a  new  database 
that  holds  ships'  measured  effectiveness  in  various  fields. 
Data  collection  started  for  all  Pacific  Fleet  ships  on  01 
September  2001  and  for  all  Atlantic  Fleet  ships  on  01  May 
2002.  Metrics  in  the  COMET  database  are  updated 
periodically.  In  the  study,  the  COMET  metrics  are  used  as 
independent  variables. 

At  the  end  of  the  Basic  Training  Phase,  ships  are 
assessed  during  FEP  according  to  mission  areas  defined  in 
COMNAVSURFORINST  3502. lA  (2003).  Each  mission  area  has 
certifications  to  be  met.  In  the  study,  the  proportion  of 
certifications  met  for  each  warfare  and  mission  area, 
representing  the  performance  of  a  ship  at  FEP,  was  defined 
to  be  the  response  variable. 

A  total  of  21  response  variables  (mission  areas 
defined  in  COMNAVSURFORINST  3502. lA  (2003))  and  44 


XV 


independent  variables  (COMET  metrics  tracked  in  COMET)  were 
used  to  build  regression  models.  There  were  51  ships  in  the 
data . 

A  logistic  regression  model  was  utilized  to  explain 
and  explore  the  effect  of  the  predictor  variables  on  each 
EEP  performance  measurement.  At  the  end  of  the  study,  it 
was  seen  that  most  of  the  response  variables  were  modeled 
only  by  the  intercept,  indicating  that  the  predictor 
variables  are  generally  not  very  helpful  for  predicting  the 
response  variables. 


XVI 


INTRODUCTION 


I . 


A.  BACKGROUND 

Aircraft  carriers  and  cruisers  deploy  overseas  for 
six-month  periods.  A  typical  battle  group  consists  of  one 
carrier,  two  assigned  Aegis  cruisers,  a  destroyer  squadron 
(four  destroyers  and  frigates) ,  two  submarines  and  an 
oiler/replenishment  ship. 

Each  operating  battle  group  typically  completes  a 
recurring  cycle  of  events  that  culminates  each  time  in 
deployment  to  the  Fifth,  Sixth  or  Seventh  Fleet.  After 
deployment,  the  Inter-Deployment  Training  Cycle  (IDTC) 
normally  begins  when  the  ship  is  transiting  to  its  homeport 
from  overseas  deployment.  After  a  leave  and  upkeep  period 
followed  by  local  at-sea  operations,  the  ship  undergoes  a 
planned  depot-level  maintenance  availability,  during  which 
the  majority  of  inter-deployment  repairs  and  equipment 
upgrades  occur.  Upon  returning  to  sea  the  ship  works  up  for 
its  next  deployment  by  completing  a  series  of  training 
exercises  and  events  which  increase  steadily  in  complexity 
as  the  crew's  operating  proficiency  increases. 

The  pre-deployment  aspects  of  the  IDTC  are  divided 
into  three  principal  phases:  basic,  intermediate  and 
advanced.  COMNAVSURFORINST  3502. lA  (2003)  describes  these 
phases  as  follows: 

1 .  Basic  Training  Phase 

The  Type  Commanders  (TYCOMs)  are  responsible  for  the 
conduct  of  Basic  Phase  Training.  The  focus  is  on  unit-level 
training  emphasizing  basic  command  and  control,  weapons 
employment,  mobility  (navigation,  seamanship,  damage 
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control,  engineering,  and  flight  operations)  and  warfare 
specialty  following  overhaul  or  major  maintenance 
availability  and  before  Change  of  Command  (CHOP)  to  the 
fleet  commander.  The  basic  training  consists  of  Command 
Assessment  of  Readiness  and  Training  II  (CART  II),  the 
Tailored  Ship' s  Training  Assessment  (TSTA) ,  Underway 
Demonstration  (UD) ,  and  the  Final  Evaluation  Problem  (FEP)  . 

2 .  Intermediate  Training  Phase 

The  Numbered  Fleet  Commanders  are  responsible  for  the 
conduct  of  intermediate  phase  training.  The  focus  in  this 
phase  is  on  warfare  team  training  and  initial  and  multi¬ 
unit  operations  under  the  traditional  Composite  Warfare 
Commander  (CWC)  concept  or  a  modified  concept  of  joint 
operations.  During  this  phase,  ships  begin  to  develop 
warfare  skills  in  coordination  with  other  units  while 
continuing  to  maintain  unit  proficiency.  The  intermediate 
training  phase  consists  of  Marine  Expeditionary  Force 
Exercise  (MEFEX)  and  Composite  Unit  Training  Exercise 
(COMPUTEX) . 

3 .  Advanced  Training  Phase 

The  focus  of  the  advanced  training,  also  under  the 
numbered  fleet  commander,  is  to  continue  to  develop  and 
refine  integrated  group  warfare  skills  and  command  and 
control  procedures  needed  to  meet  the  supported  Commander 
In  Chief's  (CINC)  specific  mission  requirements.  The 
advanced  training  phase  consists  of  the  Joint  Task  Force 
Exercise  (JTFEX) . 

The  FEP  is  the  third  command  assessment  conducted  by 
the  Immediate  Superior  in  Command  (ISIC)  and  assisted  by 
the  Afloat  Training  Group  (AFT)  that  determines  a  unit's 
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readiness  to  proceed  to  the  intermediate  and  advanced 
phases  of  the  IDTC.  The  FEP  is  conducted  subsequent  to  the 
TSTA  and  is  intended  to  demonstrate  the  ship' s  availability 
to  conduct  multiple  simultaneous  combat  missions  and 
support  functions  and  to  survive  complex  casualty  control 
situations  under  stressful  conditions,  as  well  as 
demonstrate  a  capability  to  deploy.  Table  1  is  a  pictorial 
representation  of  pre-deployment  phases  of  the  IDTC 
(COMNAVSURFORINST  3502. lA,  2003). 


BASIC 

INTERMEDIATE 

ADVANCED 

CART  II 

TSTA 

UD 

FEP 

MEFEX 

COMPUTEX 

JFTEX 

Table  1 .  Pre-Deployment  Phases  of  The  IDTC 

By  the  completion  of  FEP,  a  ship  needs  to  show  that 
she  is  "surge  ready,  "  meaning  that  she  is  ready  to  move  to 
intermediate  and  advanced  training  phases.  This  also  means 
that  she  is  immediately  deployable  as  a  unit  for  single 
operations  or  under  the  command  of  a  numbered  fleet 
commander . 

COMNAVSURFORINST  3502. lA  (2003)  defines  21  different 
warfare  and  mission  areas  and  their  standards  upon  which 
the  assessments  of  ISIC  and  AFT  will  be  based.  Each  warfare 
and  mission  area  has  criteria  that  must  be  met  by  ships  to 
be  qualified  in  that  specific  area.  Some  areas  do  not  apply 
to  some  classes  of  ships  depending  on  the  ships'  mission. 
Table  2  provides  the  list  of  the  required  certifications 
listed  by  ship  class  (COMNAVSURFORINST  3502. lA,  2003). 
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Table  2.  Required  Certifications  Listed  By  Ship  Class 


^  Not  applicable  to  Flight  (FL)  I  DDGs 
^  Vertical  Launching  System  (VLS)  ships  only 


The  number  of  cert  if  icat  ions  met  for  each  warfare  and 
mission  area  at  FEP,  according  to  the  criteria  defined  in 
the  COMNAVSURFORINST  3502. lA  (2003),  represents  the  ship's 
performance  in  that  specific  area.  The  average  number  of 
certifications  met  at  FEP  provides  an  assessment  of  the 
ship's  overall  performance  at  FEP. 

B .  OBJECTIVES 

The  purpose  of  this  thesis  is  to  find  estimators  of 
performance  of  surface  ships  at  Final  Evaluation  Problem 
(FEP)  .  If  areas  that  might  make  the  ship  fail  the  FEP  in 
the  Basic  Training  Phase  could  be  determined  before  the  FEP 
takes  place,  the  ship  could  concentrate  on  those  areas 
before  encountering  problems.  As  a  result,  this  could 
increase  a  ship's  probability  of  passing  the  FEP. 

The  COMET  metrics  were  intended  to  be  used  for 
estimation.  The  COMET  is  a  snapshot  of  a  ship's  measured 
effectiveness  in  various  fields.  Therefore,  the  main 
question  became:  "Can  we  find  estimators  among  the  COMET 
metrics  to  predict  IDTC  performances  of  ships?" 

C.  SCOPE,  LIMITATIONS  AND  ASSUMPTIONS 

The  COMET  originally  was  implemented  on  Pacific  Fleet 
surface  ships  on  01  September  2001  and  implemented  on 
Atlantic  Fleet  surface  ships  on  01  May  2002.  Being  a  new 
database,  it  has  many  missing  values  in  it.  Because  of 
these  missing  values,  some  metrics  could  not  be  included  in 
the  analysis  as  predictors. 

The  metrics  in  the  COMET  database  are  updated 
periodically  (http://extra.cnsp.navy.mil).  Therefore,  among 
those  values  which  were  taken  prior  to  the  FEP,  the  values 
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taken  closest  to  the  FEP  dates  were  used  in  this  analysis. 
The  database  of  the  FEP  dates  of  the  ships  includes  only  51 
ships.  With  this  number  the  analysis  is  possible;  however, 
more  data  points  would  give  results  with  smaller 
variability . 

As  mentioned  before,  some  warfare  and  mission  areas 
apply  to  only  specific  classes  of  ships.  Therefore,  this 
analysis  could  not  be  performed  for  some  mission  and 
warfare  areas  in  which  too  few  ships  underwent  FEP  of  that 
type . 

D.  COURSE  OF  THE  STUDY 

This  thesis  is  comprised  of  four  chapters.  Chapter  II 
describes  the  data  set  and  variable  selection.  The 
statistical  models  and  techniques  used  in  the  analysis  are 
also  explained  in  this  chapter.  Chapter  III  focuses  on  the 
logit  regression  analysis.  Chapter  IV  summarizes  the 
conclusions  of  the  analysis  and  presents  recommendations 
for  further  study. 
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II.  DATA  AND  METHODOLOGY 


A.  VARIABLE  SELECTION 

1 .  Dependent  Variables 

In  this  study,  the  dependent  variables  are  the  FEP 
performances  of  ships.  At  the  end  of  the  Basic  Training 
Phase,  ships  are  assessed  during  FEP  according  to  warfare 
and  mission  areas  listed  in  Table  2  and  defined  in 
COMNAVSURFORINST  3502. lA  (2003).  Each  warfare  and  mission 
area  has  certifications  to  be  met.  In  the  analysis,  the 
proportion  of  certifications  met  for  each  mission  area  was 
used  as  the  response  variable,  representing  the  performance 
of  a  ship  in  that  specific  area  at  FEP.  The  total 
performance  of  a  ship  was  defined  to  be  the  proportion  of 
certifications  met  for  all  areas  at  FEP.  The  AFT,  which 
assists  ISIC  to  conduct  the  FEP  assessment,  keeps  track  of 
these  data.  The  data  used  in  the  analysis  were  provided  by 
AFT  Pacific  and  consist  of  51  assessments  made  between  15 
March  2002  and  15  July  2003. 


The  Diving  and 

Salvage  Mission 

Area 

could  not 

be 

modeled  because  this 

area  was  added  to 

the 

new 

version 

of 

the  Training  Manual 

( COMNAVSURFORINST 

3502  . 

lA, 

2003) , 

and 

data  for  this  area  were  not  available. 

There  were  only  two  ships  to  which  the  Mine  Warfare 
Mission  Area  applied.  Due  to  the  lack  of  data  for  this 
area,  this  variable  could  not  be  modeled. 

2 .  Independent  Variables 

In  the  study,  the  COMET  metrics  were  used  to  represent 
independent  variables.  The  COMET  is  a  new  database  that 
keeps  ships'  measured  effectiveness  in  various  fields. 
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which  was  implemented  on  Pacific  Fleet  surface  ships  on 
01  September  2001  and  for  Atlantic  Fleet  surface  ships  on 


01  May  2002.  Metrics  in  the  COMET  database  can  be  reached 

from  Commander  Naval  Surface  Force,  Pacific  Fleet  web  page 

(http://extra.cnsp.navy.mil),  and  are  updated  online 

periodically  at  the  ship  level.  Each  metric  has  different 

periodicity.  Areas  currently  tracked  in  the  COMET  database 

are : 

a . 

Personnel 

b . 

Legal 

c . 

Dental 

d . 

Medical 

e . 

Safety 

f . 

Supply 

g. 

Maintenance  and  engineering 

h . 

Information  resources  and  combat  systems 

i  . 

Training  and  readiness 

0  • 

Warfare  readiness 

k . 

(http : / / extra 

Miscellaneous  (Ship-gathered  data) 

.cnsp.navy.mil) . 

Some  of 

the  metrics  were  excluded  because  they  were 

considered  to  be  irrelevant  to  the  analysis.  Some  metrics 
were  also  excluded  because  they  had  many  missing  values. 
The  independent  variables  and  their  descriptions  are  listed 
in  Appendix  A. 


B .  METHODOLOGY 

1.  Logit  Regression 

The  goal  of  any  regression  analysis  is  to  find  the 
best  fitting  and  most  parsimonious  and  reasonable  model  by 
which  to  describe  the  relationship  between  an  outcome 
(dependent  or  response)  variable  and  a  set  of  explanatory 
(independent  or  predictor)  variables  (Hamilton,  1992). 
Linear  regression  models  are  appropriate  for  measurement 
response  variables.  However,  many  research  questions, 
especially  in  social  and  medical  science,  involve  trying  to 
predict  whether  something  will  happen.  These  kinds  of 
questions  involve  two-category  (dichotomous)  variables, 
which  describe  whether  something  will  happen  or  will  not 
happen.  What  distinguishes  logistic  regression  models  from 
linear  regression  model  is  that  logistic  regression  handles 
dichotomous  response  variables. 

In  order  to  define  the  relationship  between  two 
measurement  variables  X  and  Y,  we  use  mathematical  models. 
One  simple  relationship  can  be  expressed  as  linear 
relationship:  Y  =  f5g  + f5jX  .  In  this  equation  Pg  represents  the 
Y  intercept  and  Pj  represents  the  slope.  According  to  this 
equation  X  is  the  predictor  of  Y.  However,  more 
realistically  in  data  analysis,  we  should  only  claim  that 
the  expected  value  of  Y  given  X  changes  linearly  with  X, 
which  can  be  expressed  as  follows  :  E\Y\X'\  =  Pg+ P^X  .  In  this 
case,  Pg  equals  the  mean  of  Y  when  X  =  0.  Since  all  models 
are  constructed  over  some  assumptions,  there  is  an  error 
embedded  in  them.  This  error  is  the  difference  between  the 
expected  value  and  the  observed  value: 
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e  =  Y-l5,-l5,X 
=  Y-E[Y\X] 


(2.1) 


Therefore,  the  actual  Y  can  be  expressed  as  the  sum  of  the 
expected  value  and  error  term: 


Y  =  E[Y\X]  +  £ 


(2.2) 


In  logistic  regression  models,  the  response  variable 
is  bounded  inside  (0,  1) .  In  order  to  model  an  outcome  with 
this  property,  the  logit  function  is  utilized: 


l(x)  =  log^ 


(2.3) 


In  this  equation,  logits  range  from  —oo  to  +o°  while 
P(x)  ranges  from  0  to  1 . 

Logit  regression  models  can  be  expressed  as  follows: 

^ )  =  ^0+  Pl^il  +  + . +  (2.4) 

In  this  model,  l{x)  is  a  continuous  linear  function  of 
predictor  variables  w,  while  P{x)  is  a  continuous 
nonlinear  function  with  an  S-shape.  P{x)  approaches,  but 
never  reaches,  the  boundaries  of  0  and  1. 

Predicted  probabilities  can  be  computed  by  inverting 
the  logit  function: 

P  =  (2.5) 

1  +  e  ^  ’ 

Logit  regression  models  have  many  of  the  desirable 
properties  of  linear  regression  models.  The  most  important 
one  is  the  linear  relationship  between  the  logits  (/(x))  and 
the  predictor  variables  (X). 
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Although  estimation  strategies  differ,  logit 
regression  requires  some  of  the  same  assumptions  as  linear 
regression  models.  Hamilton  (1992)  enumerates  these 
assumptions  as  follows: 

1.  The  model  is  specified  correctly.  For  logit 
regression  this  means  that  true  conditional 
probabilities  are  a  logistic  function  (or,  logits 
are  a  linear  function)  of  the  X  variables.  No 
important  variables  are  omitted,  and  no 
extraneous  variables  are  included.  X  variables 
are  measured  without  error. 

2.  The  cases  are  independent. 

3.  None  of  the  X  variables  are  linear  functions 
of  the  others.  Perfect  mult icollinearity  makes 
estimation  impossible;  strong  mult icollinearity 
makes  estimates  imprecise.  (Hamilton,  1992, 
p .  225) 

These  assumptions  must  be  checked  in  order  to  validate 
the  model.  According  to  Hamilton  (1992),  if  these 
conditions  are  met,  maximum  likelihood  estimates  of  the 
logit  parameters  should  theoretically  have  the  desirable 
properties  of  approximate  unbiasedness,  efficiency,  and 
normality,  in  sufficiently  large  samples. 

In  this  study,  response  variables  are  the  proportions 
of  certifications  met  for  each  warfare  area.  Since  the 
response  variables  are  restricted  to  [0,1],  logit 
regression  was  utilized,  enabling  prediction  of  the 
response  variables  without  violating  the  boundaries. 

2 .  Stepwise  Regression  and  Akaike  Information 
Criterion  (AIC) 

One  of  the  biggest  challenges  in  multiple  regression 

analysis  is  to  decide  which  predictors  are  strong 

predictors  of  the  response  variable.  Especially  after 

including  the  interactions  among  predictors  into  the  model, 
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the  set  of  predictors  can  become  huge,  and  therefore,  the 
challenge  is  harder  to  meet. 

Stepwise  search  helps  to  overcome  this  challenge. 
"Stepwise  regression"  is  a  way  of  performing  an  automated 
stepwise  search  procedure.  There  are  two  techniques  used  in 
this  procedure  (Hamilton,  1992,  p.83): 

a.  Forward  Inclusion 

The  model  starts  with  no  predictors  in  it.  At 
each  step,  the  predictor  whose  addition  improves  the  model 
most  with  respect  to  the  criterion  is  added  to  the  model. 

Jb.  Backward  Elimination 

The  model  starts  with  all  predictors  included  in 
it.  The  predictor  whose  deletion  improves  the  model  with 
respect  to  the  criterion  is  removed  at  each  step. 

The  Akaike  Information  Criterion  (AIC)  is  a 
stepwise  search  criterion  developed  to  be  utilized  in 
stepwise  regression.  The  AIC  can  be  expressed  as  follows 
(Venables  &  Ripley,  2002,  p.l74): 

AIC=  -2.l(j3,X)  +  2.p  (2.6) 

where  1{P,X)  represents  the  maximized  log  likelihood 
function  and  p  represents  the  number  of  parameters  in  the 
model.  /3  is  the  py-l  regression  parameter  vector  and  X  is 
the  nxp  design  matrix. 

In  stepwise  regression,  the  predictors  whose 
addition  will  decrease  the  AIC  most,  and  the  predictor 
whose  deletion  will  decrease  the  AIC  most,  are  found,  and 
the  step  producing  the  largest  decrease  is  taken.  If  no 
step  can  decrease  the  AIC,  the  procedure  halts.  As  can  be 
seen  from  Equation  2.6,  the  AIC  criterion  looks  for  a  good 
fit,  while  imposing  a  penalty  on  the  number  of  variables  in 
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the  model.  This  enables  the  stepwise  search  technique  to 
find  a  good-fitting  and  parsimonious  model  which,  while  not 
necessarily  optimal,  describes  the  relationship  between  the 
predictor  variables  and  response  variable. 
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Ill .  ANALYSIS 


A.  PRELIMINARY  DATA  ANALYSIS 

One  of  the  assumptions  of  logit  regression  models  is 
that  none  of  the  predictors  is  a  linear  combination  of  the 
others.  Collinearity  refers  to  a  linear  relationship 
between  two  predictor  variables  while  a  linear  relationship 
among  three  or  more  predictors  is  defined  as 
mult icollinearity .  In  the  presence  of  perfect  collinearity 
or  multicollinearity  it  is  not  possible  to  determine  the 
effect  of  a  predictor  on  the  response  variable.  Excluding 
one  of  these  variables  will  entail  no  loss  because  a 
perfect  relationship  implies  perfect  redundancy  (Hamilton, 
1992)  . 

In  order  to  detect  the  linear  relationships  among 


predictors. 

correlations  among  predictors 

and 

a  scatter 

plot  matrix 

were 

examined.  A 

relationship 

was 

defined 

as 

significant 

when 

a  correlation 

coefficient 

was 

observed 

to 

be  greater 

than 

0.6  or  less 

than  -0.6 

O 

A 

fN 

and 

Px,x2^  -0.6).  Significant  linear  relationships  among 
predictors  for  sample  size  of  51  are  as  follows: 


Predictor  Pair 

Correlation  Coefficient 

weig . main .met  -  Op. factor 

0 . 847 

main. metric  -  backlog 

-0 . 623 

repairs. 2M  -  cost.avo.2M 

0.699 

Table  3.  Significant  Linear  Relationships  Among  Predictors 
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Figure  1  is  a  scatter  plot  of  Weighted  Maintenance 
Metric  versus  Operational  Factor.  The  scatter  plot  figure 
indicates  a  positive  linear  relationship  between  these  two 
predictors  which  is  also  quantified  by  the  correlation 
coefficient  =0-847)  .  This  positive  linear  relationship 

can  be  explained  by  the  fact  that  Weighted  Maintenance 
Metric  is  calculated  as  the  product  of  Operational  Factor 
and  Maintenance  Metric.  Therefore  big  Weighted  Maintenance 
Metric  values  occur  with  big  Operational  Factor  values. 


Figure  1 . 
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Operational  Factor 

Scatter  Plot  of  Operational  Factor  vs.  Weighted 
Maintenance  Metric 


Figure  2  is  a  scatter  plot  of  Backlog  versus 
Maintenance  Metric.  Maintenance  Metric  is  calculated  as 
follows : 


Maintenance  Metric  =  —  (X  +  Y  +  Z  +  W  +  Q) 


(2.7) 


where  X  is  the  Backlog  Ratio,  Y  is  the  Self  Sufficiency 

Ratio,  Z  is  the  T/A  4  Jobs  Older  Than  Three  Months  Ratio,  W 
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is  the  Ship's  Force  and  AIMD  Costs  Ratio,  and  Q  is  the  Mean 
Total  Time  to  Correct  Ratio . 


Calculation  of  Backlog  Ratio  varies  depending  on 
whether  the  Backlog  is  above  the  Backlog  average  for  the 
ship's  class  or  below  it.  If  the  Backlog  is  above  the  class 
average,  then 


„  , ,  „  .  Backlog 

Backlog  Ratio  = - 

Class  Backlog  Average 


If  not,  then 


Backlog  Ratio  = 


^ ^  Class  Backlog  Average  ^ 
Backlog 


{-l)  +  2 


(2.8) 


(2.9) 


Both  of  these  equations  yield  positive  Backlog  Ratios 
and  they  both  decrease  when  Backlog  increases.  Backlog 
Ratio  is  one  of  the  additive  terms  in  Maintenance  Metric 
formula,  for  which,  a  small  Backlog  Ratio  is  associated 
with  a  small  Maintenance  Metric  value. 


The  correlation  coefficient  suggests  a  negative  linear 
relationship  between  these  two  variables  ( =-0 . 623)  . 

However,  the  scatter  plot  suggests  that  this  relationship 
is  nonlinear. 
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Scatter  Plot  of  Backlog  vs.  Maintenance  Metric 


Figure  3  represents  the  scatter  plot  of  Number  of  2M 
Repairs  vs.  Cost  Avoidance  2M.  The  plot  suggests  a  positive 
linear  relationship  between  these  two  predictors 
=0-699)  .  A  2M  part  is  a  Miniature/Microminiature  part 

such  as  a  transistor,  diode  and  a  resistor  which  is  used 
onboard  most  surface  ships.  This  relationship  can  be 
explained  by  the  fact  that  a  2M  cost  is  avoided  by 
repairing  a  2M  part  onboard  a  ship.  Therefore,  when  more 
parts  are  repaired  onboard  a  ship,  more  2M  cost  is  avoided. 
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Figure  3.  Scatter  Plot  of  #  of  2M  Repairs  vs.  Cost 

Avoidance  2M 

The  mult icollinearity  analysis  showed  that  there  are 
only  three  significant  mult icollinearit ies  among  the 
predictor  variables.  Therefore,  it  can  be  expected  that 
estimation  should  not  be  affected  by  mult icollinearity . 

B.  MULTIVARIATE  ANALYSIS 

Multivariate  modeling  analyzes  the  effects  of 
individual  independent  variables  on  the  response  variable 
while  holding  the  effects  of  other  variables  constant. 
Response  variables  in  the  analysis  were  the  proportions  of 
certifications  met  for  all  warfare  and  mission  areas  and 
the  total  proportion  of  certifications  met  for  all  warfare 
and  mission  areas.  Since  the  response  variables  had  values 
between  [0,  1],  multivariate  logistic  regression  techniques 

were  used  in  the  analysis. 
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The  software  package  S-Plus®  6.1  was  used  to  build 
logistic  regression  models.  A  stepwise  model  selection 
procedure  with  the  AIC  criterion  was  used  to  determine 
which  predictor  variables  and  two-way  interactions  were 
significant  in  estimating  the  response  variables.  Because 
we  sought  parsimonious  models,  the  forward  inclusion 
technique  was  used  in  stepwise  regression.  After  developing 
the  models,  diagnostics  were  checked  and  influence  analysis 
was  performed.  Table  5  summarizes  the  regression  models 
built  to  estimate  response  variables. 


RESPONSE 

VARIABLE 

COEFFICIENT 

VALUE 

STD. 

ERROR 

t  VALUE 

RESID. 

DEV. 

Aviation 

intercept 

18.249 

13.691 

1 . 332 

2 . 670 

E4 . adv 

-0 .184 

0 .163 

-1 . 131 

Amph .  Warf . 

intercept 

1 . 677 

0 .792 

2.116 

2 .800 

Anti-terror . 

Force 

Protections 

intercept 

35 . 611 

15.859 

2 .245 

10 .993 

TA4 . 3 

-0 .048 

0 . 024 

-1 . 692 

MTTT 

-0 .020 

0.009 

-2 . 101 

E4 . adv 

-0 .054 

0 . 033 

-1 . 650 

net . effect 

-0 .073 

0.046 

-1.589 

Air  Warfare 

intercept 

2.365 

0 . 504 

4  .  685 

7 .747 

Comms . 

intercept 

3 .725 

0  .  923 

4 . 032 

4.297 

Crypt . 

intercept 

3.244 

1 .097 

2 . 957 

3 .488 

EW 

intercept 

3 .038 

0  .  670 

4 . 532 

8.256 

Medical 

intercept 

69.909 

80 . 661 

0 .866 

0 .052 

dental. health 

-1 .017 

1 .199 

-0 . 848 

Intelligence 

intercept 

9 .071 

4 . 534 

2 . 000 

7.209 

gross . effect 

-0 .087 

0 .061 

-1 . 427 

Combat  Logis . 

intercept 

3 .178 

2 .282 

1 .392 

0 . 678 

Damage  Control 

intercept 

2 .968 

0  .  655 

4 . 525 

7 .149 

Engineering 

intercept 

4.274 

1 .203 

3.551 

4.219 

Navigation 

intercept 

3.337 

0 .769 

4 . 338 

7.332 
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Table  4.  Logit  Regression  Results  for  the  Response 

Variables 

Residual  deviance  is  the  measure  of  fit  used  in  logit 
regression,  which  gets  smaller  as  the  model  fits  better  to 
the  data. 

Eleven  out  of  20  models  built  by  using  logit 
regression  included  only  the  intercept  in  the  predictor 
set,  which  indicates  that,  in  those  cases,  the  COMET 
predictions  were  not  helpful  for  estimating  the  response 
variables.  The  means  of  those  response  variables  (the 
proportions  of  positive  responses)  which  had  only  the 
intercept  in  the  model  were  the  best  predictors  by  which  to 
estimate  those  variables  { (5^  equals  the  mean  of  Y  when 
X  =  0) .  Since  the  proportions  were  transformed  by  the  logit 
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function,  the  means  of  these  response  variables  are 
estimated  by: 


y= 


1 


-intercept 


) 


(2.10) 


The  reasons  that  the  COMET  predictions  were  not 
helpful  for  estimating  the  response  variables  are  the  lack 


of  data 

on 

a 

large 

number 

of 

ships  and  apparent 

unrelatedness 

of 

COMET 

metrics 

to 

FEP  assessments.  The 

criteria 

assessed 

at  FEP  are  very 

specific,  which  as  a 

result  may  not  reflect  ships'  performances  on  these  mission 
and  warfare  areas.  Most  of  the  assessments  vary  between  0.7 
and  1.0.  Additionally,  there  are  some  mission  and  warfare 
areas  for  which  all  of  the  certifications  were  met  by  all 
ships.  Without  having  variation  in  the  assessments,  it  is 
impossible  to  build  models  to  estimate  ships'  performances 
for  these  mission  and  warfare  areas. 

The  effects  of  the  predictors  on  the  response 
variables  can  be  best  examined  by  looking  at  conditional 
effect  plots.  Conditional  effect  plots  are  built  by 
plotting  one  predictor  versus  response  variable  while 
holding  the  other  predictors  at  their  mean  values.  These 
plots  show  how  a  change  in  the  value  of  a  predictor  affects 
the  response  variable.  A  linear  relationship  between 
predictors  and  logits  l{x)  implies  a  curvilinear 
relationship  between  predictors  and  response  variables 
(proportion  of  certifications  met) . 

Figure  4  is  the  conditional  effect  plot  of  Percentage 
of  Eligible  E-4  Advancements  versus  Proportion  of 
Certifications  Met  for  Aviation  Mission  Area. 
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According  to  the  conditional  effect  plot,  the  more 
eligible  E-4  advancements  a  ship  has,  the  smaller  is  the 
expected  proportion  of  certifications  met  for  the  aviation 
mission  area. 


Figure  4.  Conditional  Effect  Plot  of  %  Eligible  E-4 

Advancements  vs .  Proportion  of  Certifications  Met  for 

Aviation  Mission  Area 


The  Proportion  Met  For  the  Aviation  Mission  Area  goes 
down  most  steeply  when  percentage  of  advancements  varies 
between  70  and  100.  However,  having  eligible  E-4 
advancements  means  having  more  qualified  E-3  personnel. 
Common  intuition  says  that,  this  would  have  a  positive 
effect  on  the  Aviation  mission  area,  which  as  a  result, 
would  increase  the  certifications  met  for  this  area. 

The  Anti-Terrorism/Force  Protection  Mission  Area  was 
modeled  by  %  of  Open  2Ks  Older  than  3  Months  with  Type 

Availability  4,  Mean  Total  Time  to  Correct,  %  of  Eligible 
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Protection  Mission  Area 
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The  coefficients  of  these  predictors  are  negative, 
meaning  that  high  values  of  these  predictors  are  associated 
with  low  values  of  the  response  variable,  meaning  that,  the 
more  %  of  Open  2Ks  Older  than  3  Months  with  Type 
Availability  4,  Mean  Total  Time  to  Correct,  %  of  Eligible 
E-4  Advancements  and  Net  Effectiveness  a  ship  has,  the 
smaller  the  proportion  of  certifications  expected  to  be  met 
for  Anti/Terror  Force  Protection  Mission  Area,  s 

The  conditional  effect  plot  of  Dental  Readiness  %  is 
steeper  within  the  range  (80,  100)  .  This  means  small 
increases  in  this  predictor  variable  within  these  ranges 
are  associated  with  big  changes  in  the  Proportion  Met  For 
Anti-Terror/Force  Protection  Mission  Area. 

Medical  certification  criterion  was  modeled  by  the 
Dental  Health  %.  Figure  10  is  the  conditional  effect  plot 

of  Dental  Health  %  versus  Proportion  of  Certifications  Met 
For  Medical  Mission  Area. 
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gure  10.  Conditional  Effect  Plot  of  Dental  Health  %  vs. 

Proportion  of  Certifications  Met  for  Medical  Mission  Area 
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The  coefficient  of  this  predictor  is  -1.017,  which 
means  high  Dental  Health  %  values  are  associated  with  a  low 
proportion  value  for  the  Medical  mission  area.  This  model 
is  a  good  example  of  how  the  predictor  set  and  response 
variables  are  not  related.  According  to  COMNAVSURFORINST 
3502. lA  (2003)  there  are  no  requirements  in  the  Medical 
mission  area  regarding  the  Dental  Health  %.  Therefore,  a 
relationship  between  Medical  Mission  Area  and  Dental  Health 
%  is  not  reasonable. 

Figure  11  is  the  conditional  effect  plot  of  Gross 
Effectiveness  %  versus  Proportion  of  Certifications  Met  For 
Intelligence  Mission  Area. 
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Figure  11.  Conditional  Effect  Plot  of  Gross  Effectiveness 

vs .  Proportion  of  Certifications  Met  for  Intelligence 

Mission  Area 
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This  conditional  plot  suggests  that  the  better  the 
gross  effectiveness  a  ship  has,  the  smaller  the  number  of 
Intelligence  certifications  expected  to  be  met. 

The  Strike  Warfare  certification  criterion  was  modeled 
by  Number  of  Repairs  2M.  Figure  12  is  the  conditional 
effect  plot  of  Number  of  Repairs  2M  versus  Proportion  of 
Certifications  Met  For  the  Strike  Warfare  Area . 


Figure  12.  Conditional  Effect  Plot  of  Number  of  Repairs  2M 

vs .  Proportion  of  Certifications  Met  for  Strike  Warfare 

Certification  Criteria 


Number  of  Repairs  2M  had  a  positive  coefficient  in  the 
model.  The  conditional  effect  plot  also  reveals  that  a  high 
values  of  the  Number  of  Repairs  2M  are  associated  with  a 
high  proportion  for  the  Strike  warfare  area.  This 
relationship  appears  to  be  more  strong  within  the  range  of 
(0,  10)  repairs.  After  having  made  10  repairs,  more  repairs 
made  will  not  have  a  big  effect  on  the  proportion. 
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Undersea  warfare  area  model  was  modeled  by  the  DUI . 
Figure  13  is  the  conditional  effect  plot  of  DUI  versus 

Proportion  of  Certifications  Met  For  Undersea  Warfare  Area. 
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Figure  13.  Conditional  Effect  Plot  of  %  DUI  vs.  Proportion 

of  Certifications  Met  for  Undersea  Warfare  Certification 

Criteria 

Conditional  effect  plot  suggests  that  high  values  of 
the  %  of  DUI  are  associated  with  a  high  proportion  for 
Undersea  warfare  area.  However,  estimating  Undersea  warfare 
performance  with  %  of  DUI  is  not  a  reasonable  approach 
because  there  is  no  requirement  in  the  Undersea  warfare 
area  regarding  the  %  of  DUI  (COMNAVSURFORINST  3502. lA, 
2003,  p.  2-4-S-l) . 

Visit,  Board,  Search  and  Seizure  mission  area  was 
modeled  by  %  of  SSEW  Billets  Complete,  %  of  Eligible  E-5 
Advancements,  %  FPFT  Billets  Complete  and  Reenlistment  % 
Zone  B.  Figures  14,  15,  16  and  17  are  conditional  effect 

plots  of  these  predictor  variables. 
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Among  the  predictors,  %  of  Eligible  E-5  Advancements 
and  Reenlistment  %  Zone  B  have  negative  coefficients  while 
%  of  SSEW  Billets  Complete  and  %  FPFT  Billets  Complete  have 
positive  coefficients.  Therefore,  high  values  of  the  %  of 

Eligible  E-5  Advancements  or  the  Reenlistment  %  Zone  B  are 

associated  with  a  low  proportion  for  Visit,  Board,  Search 
and  Seizure  mission  area,  while  high  proportion  for  Visit, 
Board,  Search  and  Seizure  mission  area  is  associated  with 
high  values  of  the  latter  two. 

The  conditional  effect  curves  of  %  SSEW  Billets 
Complete,  %  FPFT  Billets  Complete  and  Reenlistment  %  Zone  B 

are  not  steep,  which  suggest  that  big  increases  in  the 
values  of  %  SSEW  Billets  Complete,  %  FPFT  Billets  Complete 
and  Reenlistment  %  Zone  B  change  are  associated  with  small 
decreases  in  the  proportion  met  for  Visit,  Board,  Search 
and  Seizure  Mission  Area.  However,  the  conditional  effect 
curve  of  %  Eligible  E-5  Advancements  is  much  steeper, 
especially  within  the  ranges  (40,80).  Therefore,  a  small 
decrease  in  the  %  Eligible  E-5  Advancements  within  this 
range  is  associated  with  a  big  increase  in  the  Proportion 
Met  for  Visit,  Board,  Search  and  Seizure  Mission  Area. 

Force  Maintenance  and  Material  Management  mission  area 
was  modeled  by  Commander  Naval  Surface  Forces  (CNSF)  Drug 
Testing  Standard  Unit  Performance  and  Gross  Effectiveness . 

Conditional  effect  plots  of  these  variables  are  as  follows: 
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Both  of  these  variables  have  positive  coefficients, 
which  mean  high  values  of  these  variables  are  associated 
with  high  proportion  for  Force  Maintenance  and  Material 
Management  mission  area.  The  curves  are  much  steeper  within 
ranges  (23,24)  for  Drug  Testing  Unit  Performance  and  within 
ranges  (53,57)  for  Gross  Effectiveness,  meaning  that  small 
increases  within  these  ranges  are  associated  with  big 
increases  in  the  Proportion  of  Certifications  Met  For  Force 
Maintenance  and  Material  Management  Mission  Area.  On  the 
other  hand  there  are  only  13  observations  for  this  mission 
area;  therefore,  a  small  sample  size  like  this  may  not 
yield  reasonable  inferences.  Both  conditional  effect  plots 
show  a  sigmoidal  shape.  The  reason  for  this  is  that  there 
are  only  two  values  in  the  response  variable:  1.0  and  0.0. 

The  proportion  of  total  number  of  certifications  met 
for  all  warfare  and  mission  areas  represents  ships'  total 
performance  at  FEP .  This  response  variable  was  modeled  by 
DUI .  Figure  29  is  the  conditional  effect  plot  of  %  of  DUI 
versus  Proportion  of  Total  Number  of  Certifications  Met . 


Figure  20.  Conditional  Effect  Plot  of  %  of  DUI  vs. 

Proportion  of  Total  Number  of  Certifications  Met 
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The  fact  that  Proportion  of  Total  Number  of 
Certifications  Met  was  modeled  by  DUI  only  is  also  a  good 
indication  of  anomalies  in  the  data.  None  of  the  mission 
areas  includes  a  criterion  regarding  the  DUI 
(COMNAVSURFORINST,  2003,  pp .  2-4-A-l  -  2-4-U-4)  . 

Additionally,  %  DUI  varies  between  (0,2),  which  is  a  very 
small  interval  indicating  that  variation  in  this  metric  is 
very  small.  This  model,  like  some  other  models,  does  not 
provide  a  reasonable  estimating  technique. 

C.  INFLUENCE  ANALYSIS 

A  case  is  influential  if  its  deletion  substantially 
changes  the  regression  results.  Influential  cases  are  not 
necessarily  outliers,  influence  results  from  a  particular 
combination  of  values  on  all  variables  in  the  regression.  In 
order  to  find  the  influential  cases,  the  A5  statistic  which 
measures  the  standardized  change  in  estimated  parameters 
that  result  from  deleting  all  cases  with  the  jth  X 

pattern  was  considered.  AS^.  can  be  expressed  as  follows: 

r/h: 

AB,=  ^  \  (2.11) 

where  h-  is  the  leverage  of  the  jth  case  and  is  the 

Pearson  residual.  A  large  value  of  AB-  indicates  that  the 
jth  pattern  exerts  substantial  influence.  According  to 
Hamilton  (1992,  p.236),  case  j  is  influential  if  AB->1. 

Plots  of  AB  of  mission  and  warfare  areas  were  examined 
to  detect  influential  cases.  Two  models  with  highly 
influential  cases  were  found  in  the  data.  Having  a  AB  value 
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of  5.52,  USS  PEARL  HARBOR  was  the  most  influential  case  in 
the  Medical  mission  area  model.  A  graph  of  A5  is  shown  in 
Figure  2 1 . 
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Figure  21.  Influence  Statistic  AS  vs.  Ships  for  Medical 

Mission  Area 

The  USS  PEARL  HARBOR  was  the  most  influential  case  in 
the  Medical  mission  area  because  she  is  the  only  ship 
having  a  proportion  other  than  1.0  (her  value  is  0.67) .  The 
rest  of  the  ships  have  a  medical  proportion  of  1.0. 
Therefore,  deleting  her  from  the  model  makes  the  model 
invalid  and  making  predictions  impossible  because  in  that 
case  all  of  the  proportions  for  this  mission  area  would  be 
1.0. 

The  other  influential  case  was  the  USS  LAKE  ERIE  in 
the  Force  Maintenance  and  Material  Management  mission  area 
model.  It  had  a  AS  statistic  of  6.574.  Figure  22  represents 
the  influence  diagnostic  graph  for  the  Force  Maintenance 
and  Material  Management  mission  area. 
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Figure  22.  Influence  Statistic  AS  vs.  Ships  for  Force 

Maintenance  and  Material  Management  Mission  Area 
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Figure  2 3 . 


This  diagnostic  graph  suggests  that  Force  Maintenance 
and  Material  Management  mission  area  could  have  been 
modeled  with  Gross  Effectiveness  only  if  the  data  had  not 
included  the  USS  LAKE  ERIE  in  this  mission  area.  Then  it 
would  have  been  possible  to  suggest  that  a  ship  whose  Gross 
Effectiveness  value  is  more  than  59  is  likely  to  achieve 
1.0  effectiveness  proportion  for  Force  Maintenance  and 
Material  Management  mission  area  at  FEP . 

Similarly,  without  having  the  USS  LAKE  CHAMPLAIN  in 
data,  it  would  have  been  possible  to  model  Eorce 
Maintenance  and  Material  Management  mission  area  only  with 
Drug  Testing  Unit  Performance.  This  would  have  yielded  a 
conclusion  that  a  ship  whose  Drug  Testing  Unit  Performance 
is  greater  than  24.5  is  likely  to  achieve  a  1.0  proportion 
for  the  Eorce  Maintenance  and  Material  Management  mission 
area  at  EEP . 

There  are  two  reasons  for  having  these  influential 
cases.  One  of  them  is  that  there  are  not  enough  data  points 
to  make  strong  inferences  about  the  models  and  cases.  When 
the  number  of  data  points  increases,  the  influential  cases 
tend  to  have  less  influence  on  the  model  due  to  variation. 
The  other  reason  is  that  proportions  met  for  most  of  the 
mission  and  warfare  areas  tend  to  be  close  to  1.0.  As  a 
result,  a  ship  having  a  smaller  proportion  for  a  mission 
and  warfare  area  compared  to  other  ships  makes  a 
substantial  influence  on  the  model. 
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IV.  SUMMARY,  LIMITATIONS  AND  RECOMMENDATIONS 


The  purpose  of  this  thesis  was  to  estimate  IDTC 
performances  of  the  US  Pacific  Fleet  surface  ships,  which 
are  evaluated  at  the  end  of  the  Basic  Training  Phase,  by 
using  the  COMET  metrics.  Proportions  of  certifications  met 
for  each  mission  and  warfare  area  were  used  as  indicators 
of  performances  of  ships  at  FEP .  Therefore,  in  the 
analysis,  these  were  the  response  variables  which  were 
intended  to  be  modeled  by  using  regression  models.  The 
metrics  in  the  COMET  data  base  were  used  as  independent 
variables  by  which  to  estimate  the  response  variables. 
Since  the  response  variables  are  bounded  in  [0,1],  logistic 
regression  was  used  to  build  models.  In  order  to  determine 
which  predictor  variables  were  strong  estimators  of  the 
response  variables,  stepwise  regression  techniques  were 
utilized.  The  AIC  was  used  to  determine  independent 
variables  and  two-way  interactions  to  be  added  and  deleted. 
Having  developed  the  models,  influence  analysis  was 
performed  and  diagnostics  were  checked. 

Eleven  out  of  20  models  built  by  using  multivariate 
logistic  regression  model  included  only  the  intercept  in 
the  predictor  set.  The  other  nine  models  included  at  most 
four  independent  variables  as  predictors  for  the  response 


variables.  Considering 

that 

the 

data  set  included 
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independent  variables. 
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are 

not 

helpful  to  estimate 

EEP 

performances  of  ships. 

The  data  consist  of  44 

metrics 

from  the  COMET 

data 

base  and  22  assessments 

from 

the 

EEP 

(21  for  mission 

and 

warfare  areas,  one  for  proportion  of  total  number  of 


41 


cert if icat ions  met  for  all  mission  and  warfare  areas)  for 
51  ships.  The  number  of  ships  in  the  data  was  sufficient 
enough  to  perform  the  analysis;  however,  data  for  more 
ships  might  yield  more  reasonable  answers. 

Each  warfare  and  mission  area  has  criteria  that  must 
be  met  by  ships  in  order  to  be  qualified  in  that  area.  Some 
areas  do  not  apply  to  all  classes  of  ships  based  on 
mission.  The  Diving  and  Salvage  mission  area  could  not  be 
modeled  because  this  area  was  added  to  the  new  version  of 
the  Training  Manual  (COMNAVSURFORINST  3502. lA,  2003)  which 
was  effective  on  07  April  2003,  and  data  for  this  area  was 
not  available  due  to  the  lack  of  ARS  type  of  ships  having 
undergone  FEP  up  to  the  time  of  the  analysis.  Additionally, 
there  were  only  two  ships  to  which  the  Mine  Warfare  mission 
area  applied.  Due  to  the  lack  of  data  for  this  area,  this 
area  could  not  be  modeled  as  well. 

According  to  COMNAVSURFORINST  3502. lA  (2003),  almost 
all  of  the  mission  and  warfare  areas  have,  as  a 
certification  requirement,  that  all  relevant  personnel 
complete  (or  have  a  plan  for  completion  of)  schooling 
required  for  that  area.  However,  none  of  the  models 
included  FPFT  (%  Force  Protection  Fundamentals  Training 
Billets  Complete)  and  IBFT  (Integrated  Battle  Force 
Training  %  Complete)  as  predictors.  Normally,  these  metrics 
would  be  expected  to  be  in  the  predictor  sets  of  most  of 
the  models. 

Even  if  some  of  the  predictor  variables  seem  to  be 

irrelevant  for  a  mission  and  warfare  area  or  the  mission 

and  warfare  area  does  not  have  a  requirement  regarding 

these  kinds  of  predictors,  some  of  the  areas  were  modeled 

by  these  irrelevant  predictor  variables.  The  reason  for 

this  is  that  the  requirements  for  mission  and  warfare  areas 
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are  very  specific,  which  as  a  result  may  not  reflect  a 
ship's  performance  for  that  area. 

Most  of  the  assessments  for  mission  and  warfare  areas 
tend  to  range  from  0.7  to  1.0.  Additionally,  there  are  some 
mission  and  warfare  areas  for  which  all  of  the 
certifications  are  met  by  all  ships.  Therefore,  most  of  the 
response  variables'  means  are  very  high;  some  are  even  1.0. 
Without  having  variation  in  the  assessments,  it  is 
impossible  to  build  models  to  estimate  ships'  performances 
for  these  mission  and  warfare  areas. 

In  the  future,  this  kind  of  an  analysis  can  be 
performed  by  using  the  COMET  database  and  using  some  other 
MOP  (Measure  Of  Performance) .  Also  including  US  Atlantic 
Fleet  Surface  ships  will  increase  the  size  of  the  data, 
which  would  help  the  analysis.  The  biggest  challenge  in 
this  analysis  is  to  find  the  best  MOP  by  which  to  represent 
ships'  IDTC  performances.  The  assessments  based  on  grades 
in  those  mission  and  warfare  areas  would  be  the  best  MOP  to 
estimate  and  to  build  models. 


43 


THIS  PAGE  INTENTIONALLY  LEE!  BLANK 


44 


APPENDIX  A.  INDEPENDENT  VARIABLES 


VARIABLE  NAME 

VARIABLE 

DESCRIPTION 

DLR . oblig 

Current  FY  DLR 
%  Obligation 

The  percentage  that  Depot  Level 

Repairable  (DLR)  carcass  charges  (the 
charge  for  not  returning  the  broken  unit 
to  the  repair  facility)  comprise  of  the 
total  DLR  obligations  (money  spent)  for  a 
given  Fiscal  Year  (FY) . 

DUI 

%  DUI's 

Average  percentage  of  ship' s  crew  that 
were  adjudged  a  Driving  Under  Influence 
(DUI)  over  the  last  four  quarters. 

E4  .  adv 

%  of  Eligible 

E-4 

Advancements 

The  percentage  of  rank  E-3  crew  eligible 
to  advance  to  rank  E-4 . 

E5 . LTC . comp 

E-5  LTC 
Completion 

The  percentage  of  rank  E-5  crew  completed 
Leadership  Training  Course. 

E5 . adv 

%  of  Eligible 

E-5 

Advancements 

The  percentage  of  rank  E-4  crew  eligible 
to  advance  to  rank  E-5. 

E6 . LTC . comp 

E-6  LTC 
Completion 

The  percentage  of  rank  E-6  crew  completed 
Leadership  Training  Course. 

E6 . adv 

%  of  Eligible 

E-6 

Advancements 

The  percentage  of  rank  E-5  crew  eligible 
to  advance  to  rank  E-6. 

E7 . LTC . comp 

E-7  LTC 
Completion 

The  percentage  of  rank  E-7  crew  completed 
Leadership  Training  Course. 

E7 . adv 

%  of  Eligible 

E-7 

Advancements 

The  percentage  of  rank  E-6  crew  eligible 
to  advance  to  rank  E-7 . 

ESWS 

%  ESWS 

Qualified 

Sailors 

The  percentage  of  Enlisted  Surface 

Warfare  Specialist  (ESWS)  recipients 
among  all  the  sailors  that  are  required 
to  be  qualified. 

FPFT 

%  FPFT  Billets 
Complete 

The  percentage  of  all  Force  Protection 
Fundamentals  Training  (FPFT)  requirements 
completed.  Each  ship  class  has  different 
requirement  for  this  training. 

IBFT 

IBFT  %  Complete 

The  percentage  of  all  Integrated  Battle 
Force  Training  (IBFT)  requirements 
completed . 

OP  .  factor 

Op  Factor 

Operational  Factor.  Represents  a  ship's 
operation  tempo  or  underway  time.  Big 
values  of  Operational  Factor  are 
associated  with  long  underway  times. 
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SSEW 

%  SSEW  Billets 
Complete 

The  percentage  of  all  Shipboard  Security 
Engagement  Weapons  (SSEW)  Training 
requirements  completed.  Each  ship  class 
has  different  requirement  for  this 
training . 

TA .  4 . 3 

T/A  4>3  Months 

The  percentage  of  open  2Ks  older  than  3 
months  with  type  availability  4.  2K  is  a 
maintenance  form  generated  when  a 
discrepancy  is  observed  onboard  a  ship. 

Type  availability  4  refers  to  the 
maintenance  performed  by  the  ship's  crew. 

Uri . sweep 

#  of  Urinalysis 
Sweep 

Unit  sweeps  include  drug  testing  of  all 
personnel  assigned  to  the  command.  The 
urinalysis  data  is  a  rolling  12-month 
window  for  the  period  June  2002-May  2003. 
This  is  the  total  samples  divided  by  the 
Current  On  Board  (COB) .  For  each  multiple 
of  70%  the  ship  gets  credit  for  1  unit 
sweep . 

ZoneA . att 

Attrition  % 

Zone  A 

The  percentage  of  attrition  in  Zone  A. 

Zone  A  refers  to  sailors  with  0  to  6 
years  of  active  service. 

ZoneA . reen 

Reenlistment  % 

Zone  A 

The  percentage  of  reenlistment  in  Zone  A. 
Zone  A  refers  to  sailors  with  0  to  6 
years  of  active  service. 

ZoneB . reen 

Reenlistment  % 

Zone  B 

The  percentage  of  reenlistment  in  Zone  B. 
Zone  B  refers  to  sailors  with  7  to  10 
years  of  active  service. 

ZoneC . reen 

Reenlistment  % 

Zone  C 

The  percentage  of  reenlistment  in  Zone  C. 
Zone  C  refers  to  sailors  with  11  to  14 
years  of  active  service. 

Backlog 

Backlog 

The  number  of  deferred  failures  in  a 
ship' s  Consolidated  Ships  Maintenance 
Project  (CSMP) . 

form . cour . req 

#  of  Unfilled 
Formal  Course 
Requirements 

Number  of  unfilled  school  quotas  required 
by  the  Surface  Training  Manual. 

com . cal . read 

Combat 

Calibration 

Readiness 

The  percentage  of  shipboard 
general/special  purpose  electronic  test 
equipment  current  in  regards  to  their 
calibration  periodicity. 

com . pack 

#  of 

Commissioning 

Packages 

The  number  of  commissioning  packages 
submitted  during  the  current  fiscal  year. 

comp . col . cour 

%  of  Crew 
Completing 
College  Courses 

Total  number  of  college  courses  completed 
in  fiscal  year  2003. 
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cost . avo . 2M 

2M  Cost 

Avoidance 

Total  cost  avoided  by  repairing  2M 
(Miniature/  Microminiature)  parts  onboard 
the  ship  in  a  quarter. 

dental . health 

Dental  Health  % 

The  percentage  of  the  crew  having  Class  1 
dental  status.  Personnel  with  a  dental 
status  of  either  Class  1  or  2  have 
minimal  dental  disease  conditions  and  are 
considered  ready  for  deployments  where 
dental  support  may  not  be  available. 

dental . read 

Dental 

Readiness  % 

The  percentage  of  the  crew  with  a  Class  1 
or  2  dental  status.  Personnel  with  a 
dental  status  of  either  Class  1  or  2  have 
minimal  dental  disease  conditions  and  are 
considered  ready  for  deployments  where 
dental  support  may  not  be  available. 

drug . test 

Commander  Naval 

Surface  Forces 
(CNSF)  Drug 
Testing 

Standard  Unit 

Performance 

Total  number  of  urinalysis  sweeps  divided 
by  the  COB  for  each  month.  This  is 
totaled  and  divided  by  12  to  determine 
the  rolling  12  month  average  for  the 
period  June  2002-May  2003. 

eng . cal . read 

Engineering 

Calibration 

Readiness 

The  percentage  of  shipboard  engineering 
gages,  meters,  and  associated  Shipboard 
Gage  Calibration  Program  (SGCP)  equipment 
current  regards  to  their  calibration 
periodicity . 

fail . col . cour 

%  of  Crew 

Failing  College 
Courses 

The  total  number  of  college  course 
failures  for  all  the  Navy  College  Program 
for  Afloat  College  Education  (NCPACE) 
offered  courses  on  the  ship  in  fiscal 
year  2003. 

for . IMA . cost 

Ship  Force  and 
IMA  Costs 

18  month  average  of  the  total 

Intermediate  Maintenance  Activity  (IMA) 
man-hours  times  the  IMA  labor  rate  plus 
the  Total  Replacement  Cost  of  all  repair 
parts  ordered.  IMA  is  a  military 
organization  specializes  in  the  repair  of 
certain  pieces  of  equipment. 

gendets 

#  General 

Detail  Sailors 
(GENDETS)  to 
A-School 

The  number  of  undesignated  permanent 
ship's  crew  sent  to  an  A-School.  A-School 
Core  courses  include  general  Icnowledge 
and  skills  training  for  the  particular 
rating,  while  A-School  Strand  courses 
focus  on  the  more  specialized  training 
requirements  for  that  rating  and  a 
specific  aircraft  or  equipment,  based  on 
the  student's  fleet  activity  destination. 

gross . effect 

Gross 

Effectiveness  % 

Issue  percentage  of  repair  parts  from 
stock  base  on  total  demands.  The  low- 
gross  effectiveness  means  that  the  range 
of  stock  is  insufficient. 
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Comp . col . cour 

%  of  Crew 
Completing 
College  Courses 

Total  number  of  NCPACE  delivered  college 
courses  completed  in  fiscal  year  2003. 

maint . metric 

Maintenance 

Metric 

This  metric  is  calculated  by  summing  Self 
Sufficiency  Ratio,  Mean  Total  Time  to 
Correct  Ratio,  Ships  Force  and  IMA  Costs 
Ratio,  Backlog  Ratio,  and  T/A  4>6  Months 
metric  and  then  dividing  this  value  by  5. 
Number  is  scaled  between  0  and  2.  High 
values  of  this  metric  are  desired. 

mast . cases 

%  Mast  Cases 

Average  percentage  of  the  ship' s  crew 
that  received  Non-Judicial  Punishment 
(NJP)  over  the  last  four  quarters. 

net . effect 

Net 

Effectiveness  % 

Issue  percentage  of  repair  parts  from 
stock.  The  low-net  effectiveness  means 
the  depth  of  stock  is  insufficient. 

random . comp 

Monthly  Random 
Compliance 

Urinalysis  Sweep  of  randomly  selected 
personnel  on  board.  This  is  the  total 
number  of  samples  divided  by  COB  for  the 
most  recent  month.  If  this  is  greater 
than  CNSF  standard  of  20%,  then  the  ship 
is  in  compliance. 

repairs . 2M 

#  of  Repairs  2M 

Total  number  of  2M  parts  repaired  onboard 
the  ship  in  a  quarter. 

self . suf f 

Self 

Sufficiency 

18  month  average  of  Status  2  and  3 
failures  corrected  by  ship' s  force 
divided  by  the  total  number  of  Status  2 
and  3  failures  corrected  by  ship's  force. 
Status  2  failure  is  one  that  results  in 
the  equipment  being  inoperable.  Status  3 
failure  is  one  that  results  in  the 
degradation  of  the  equipments  capability. 

SWO 

%  SWO  Qualified 
Officers 

The  percentage  of  Surface  Warfare  Officer 
(SWO)  qualified  officers  among  all  the 
officers  required  to  be  SWO  qualified. 

MTTT 

Mean  Total  Time 

to  Correct 

18  month  average  of  calendar  days  between 
discovery  of  the  failure  and  the 
completion  of  the  2K.  2K  is  a  maintenance 
form  generated  when  a  discrepancy  is 
observed  onboard  a  ship. 

weig . main . met 

Weighted 

Maintenance 

Metric 

Maintenance  metric  multiplied  by  the  Op 
Factor . 

Table  5.  Independent  Variables  and  Descriptions 
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